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(57) ABSTRACT 

Reception of digital multimedia data signals from multiple 
sources in a stream over a fixed bandwidth communications 
path may be accomplished by subscribing to a base layer of 
a first source and a base layer of a second source, and 
subscribing to an enhancement layer of the first source. Data 
signals corresponding to the subscribed layers of the first 
and second sources may be received in a stream over the 
fixed bandwidth communications path, output signals may 
be produced which correspond to the received data signals 
for the first source, and output signals may be produced 
corresponding to the received data signals for the second 
source. The output signals for the first source may be 
displayed in a first portion or window of a display, and 
output signals for the second source may be simultaneously 
displayed in a second portion or window of the display, 
thereby providing a picture -in-picture (PIP) display for 
streaming digital video. 

19 Claims, 5 Drawing Sheets 
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METHOD OF STREAMING VIDEO FROM with a purported maximum bandwidth of 56K bytes per 

MULTIPLE SOURCES OVER A NETWORK second (bps), the modem may serve as an impediment to 

providing a clearer or larger display of the video images. 
This limitation of a fixed bandwidth communications path 
BACKGROUND $ ^ particularly acute when the client desires to retrieve 
1. Field streams from more than one signal source at a time. In some 
The present invention relates generally to processing of situations, a user may desire to display video data signals 
multimedia data signals and, more specifically, to manage- from at least ^ sources on a single display, with the 
ment of streaming digital audio and video data signals. data &° m cach sourcc bcin S represented on a portion of the 
2 Description 10 ^ispky* Th e display °f two independent video streams, each 
. . . , in a portion of a display, is widely used for analog television 
Digital multimedia data sjgnals may be communicated si ^ „ ^ -^cm^.pi^" (PIP) feature of many 
over a network from a signal source to a client system for ^ televisions. However, for analog televisions, each 
viewing and bstemng by a user at the client site. In one ^ fc received fe a ^ ^ Qn a different 
scenario, a user requests the retrieval of a sequence :of audio 15 comlnumcauons th ^ the bandwidth of a 
and video data signals called a stream from the s,gnal sm eamBgu ^ ll&aa th for me toalsomtxs is not fixed, 
source, which may be a server system communicating with ~\ _ , , 
the client system over the network. When the audio and c Simultaneous reception of two or more streams over a 
video data signals making up the stream are received at the fixed > ^™ dth ' ^ eommunicatinns path may 
client, these multimedia data signals are processed to pro- „ result m bandwidui conte^nUon between the streams. Without 
j i * j * c it, • *u t f a a» a some way to prioritize the client s reception of the streams, 
duce output data for the user m the form of sounds produced , 7 F , , , , 
. _ „ tt irtn*i«.«a i,.„ „„ A nn „ the streams may contend for the available, limited band- 
by one or more loudspeakers and images displayed on a . , , _ t J . . . ' . , 
, width. The data packets from the streams are thus received 

5, _ . 1- ina non-deterministic manner. The result may be that the 

The concept of streammg multimedia includes a muto- doa and processing of the streams fc degraded to the 

media source sending a multimedia stream to a multimedia M ^ wbcre ^ o{ ^ strcams m significant 

recipient for real-umepresentauon to auser. That is, stream- ^ Motm!ltion t0 me user TypicaU y, the user cannot 

mg can be viewed as a display/play model as opposed to sdcct which strcam tQ rec6iv6 ^ m mc 

a more traditional download and display/play model, ^ n to ensur<; ^ at ^ Qne of me $tKims ^ 

where the eutrre stream is collected by the recipient prior to fa da(a A ^ for a suitabk ^ tQ , he ^ Jn 

presentation. Successful streammg is based on the premise 30 addition> ^ of audio data si ^ for a stream ma be 

that the multimedia preparation (e.g., compression) and the d ded such ^ ^ audfa Qu ^ 

transmission mechamsm can feed the recipient at a rate fast and undesillble for me ^ 

enough for the multimedia stream to be presented in real- „_ A . , f . , A t , , 

, u rr • „ t ftU What is needed is an approach or technique that addresses 

time. Streammg may involve buffering a small amount of the . ki 

stream prior to presentation, but the amount of buffering is 35 e ore S om 5 P ro ems - 

typicaDy small compared to the length of the entire stream. SUMMARY 

The rationale for buffering is typically to smooth out jitter 

inherent in network-based transmission mechanisms (e.g., An embodiment of the present invention is a method of 

packet-based transports, etc.). receiving digital multimedia data signals from multiple 

^ u *. « * • „ A - m A 1 An sources over a fixed bandwidth communications path using 

One difference between "streaming multimedia and mul- 40 . *7 ... & 

^ j- .l * l 1. j 1 j j /■ * * 4* « \ multiple layers of a stream. The method includes subscribing 

umedia that has been downloaded (in part or in its entirety) £ , c . a * 1 f 5 

. , . „• „ w~~a „ „„.t;«, to a first layer of a first source, a first layer of a second 

pnor to presentation is that streaming multimedia quality , J A t , . ' A r™ if _ , 

r /c r , , . /. - \ • j * • j * source, and a second layer of the first source. The method 

(for example, video clarity and frame -rate) is determined, at . . , 1 

, 4 . j j«u „ u further mcludes receivme data signals corresponding to the 

least m part, by the bandwidth of the transmission mecha- , n , . * * * j j • ,l 

1- j 1 a a w a- *u I * f a subscribed layers of the first and second sources m the 

msm. For downloaded multimedia, the quality 01 the pre- 45 ( J . , , , . , , . . , 

sentation is typically unaltered from what it had been st ™ m over me f xed ^width commimications path and 

authored as; however, the download times are affected by the Pjoducing output signals correspondmg to the received data 

bandwidth of the transmission channel. Asecond difference S1 § nals for the fet source and me second source ' 

between the two models is that the download model is not Another embodiment of the present invention is a method 

workable for ( live" content. In a display/play model of a 50 of receiving multiple, independent digital multimedia 

digital multimedia streaming system, the size of the dis- streams over a bandwidth communications path. The 

played images and the overall quality of the audio and video method includes subscribing to first and second streams, the 

presented to the user are limited at least in part by the first stream corresponding to a first multimedia content and 

bandwidth of the communications path between the signal mc second stream corresponding to a second multimedia 

source and the client. In some systems, a sufficient amount 55 content. The method further includes concurrently receiving 

of audio and video data signals is received to display data signals corresponding to the streams over the fixed 

full-motion video and concurrent audio to the user in a bandwidth communications path, and producing output sig- 

real-time (or streaming) mode, although the size and quality nals corresponding to the received data signals for the first 

of the displayed image may be inadequate or undesirable for stream and for the second stream, 

some uses. If the bandwidth of the communications path 60 BRIEF DESCRIPTION OF THE DRAWINGS 
could be expanded, then a larger image, a clearer image, or 

a smoother set of images could be presented to the user. The features and advantages of the present invention will 

However, in most systems, the communications path become apparent from the following detailed description of 

between the signal source and the client is a fixed, relatively the present invention in which: 

low bandwidth communication channel having at least one 65 FIG. 1 is a diagram of a system for multiple source 

bottleneck. For example, when the network is the Internet real-time video streaming over a network according to one 

and the client is a computer system employing a modem embodiment of the present invention; 
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FIG. 2 is a diagram of a system for dual source real-time is not limited in this respect The connection between 

video streaming over a network according to another communications network 14 and multimedia access and 

embodiment of the present invention; display device 12 represented by communications path 16 

HG.3isadiagramiUustratmgbanQ^dm P arUtionmgof J™*, »?e any link with a predefined maximum data 

a communication* path and corresponding video images s bandwidth, such as for examp e modem-based dud-up 

~7r . m < • . r.u . * connections, various types of digital subscriber hne (DSL) 

according to one embodiment of the present invention; connectLor ; S) cable modem connections, wireless 

FIG. 4 is a flow diagram of controlling bandwidth parti- connections, or network interface card (NIC) based intranet 

tioning of multiple layers of a stream according to an connections, although the invention is not limited in this 

embodiment of the present invention; and ^ respect. For purposes of this embodiment, only a fixed 

FIG. 5 is a diagram illustrating a sample system suitable maximum bandwidth for communications path 16 is 

to be programmed according to an embodiment of a method stipulated, however, the actual data rate for the communi- 

for streaming video from multiple sources over a network. cations path may vary with time without exceeding the 

maximum value. Additionally, the communications path 

DETAILED DESCRIPTION ^ supports bi-directional data flow (although not necessarily 

An embodiment of the present invention includes a symmetrical in terms of bandwidth for each direction of data 

method whereby a client receives at least two real-time flow). 

independent streams or at least two layers of a stream A plurality of signal sources Source 1 18, Source 
containing digital video data signals over a fixed bandwidth 2 20, ... , Source N 22 may be coupled to communications 
communications path with user control of which stream or 20 network 14 to serve as multimedia content repositories 
layer is "in focus" as the primary source of display data and accessed by multimedia access and display device 12. In one 
which stream or layer is a secondary source for display data. embodiment, the number of signal sources actively serving 
In one embodiment, the method may be accomplished using multimedia access and display device 12 may be two. In 
scaleable audio and video coding. An embodiment of the other embodiments, any number of signal sources may be 
present invention provides a picture-in-picture (PIP) capa- 25 active and available for serving multimedia data streams to 
bility to live video streaming over a communications chan- multimedia access and display devices at any time. Each 
nel with limited bandwidth (such as a dial-up Internet signal source stores or has access to at least a portion of a 
connection, for example). This enables new functions for digital multimedia content represented as a sequence of 
network streaming video applications (such as Internet audio and video data signals called streams. The multimedia 
streaming, for example) and may be used to improve stream- 30 content may be stored at a signal source site in a main 
ing applications that incorporate a greater degree of user- memory, a secondary memory, or a storage medium such as, 
interactivity and improved usability. for example, a CD-ROM, a digital versatile disk (DVD), or 
In one embodiment of the present invention, a client other suitable apparatus. As used herein, a stream comprises 
system may receive a single stream of video data signals, the a plurality of audio and/or video data signals and the stream 
single stream comprising multiple layers, each layer being 35 ma y comprise multiple layers. Each layer of a stream may 
communicated from a different video data source and con- comprise data from a source. The multimedia content may 
taining video data signals of a portion of a selected multi- also be acquired in real-time from live audio and/or video 
media content. The client system may present video data sources such as, for example, a video camera and a micro- 
signals from a first source for display in a primary portion of phone. 

a display while simultaneously displaying video data signals 40 When a user of multimedia access and display device 12 
from a second source in a secondary portion of the display. desires to hear and/or view selected multimedia content, the 
At the direction of the user via a graphical user interface, the user indicates bis or her selection to multimedia access and 
client system may then switch the bandwidth partitioning of display device 12 in any one of many well-known ways, 
the stream so that video data signals from the first source such as, for example, by using a pointing device or keyboard 
may be displayed in the secondary portion of the display and 45 and a graphical user interface and/or a browser application, 
video data signals from the second source may be displayed In one embodiment, specific desired multimedia content 
in the primary portion of the display. The video data signals may be identified by a uniform resource locator (URL) 
from each of the sources may be carried by layers of the and/or identifiers according to the well-known hyper text 
stream and may be received over the same limited band- transport protocol (HTTP), although the invention is not 
width communications path. However, the allocation of the 50 limited in scope in this respect. Multimedia access and 
bandwidth between the primary and secondary sources display device 12 interacts with a signal source having the 
changes according to the user selection. desired content to initiate the sending of a stream of audio 
FIG. 1 is a diagram of a system 10 for multiple source and/or video data signals representing the desired content 
real-time video streaming over a network according to one over communications network 14. Upon reception, multi- 
embodiment of the present invention. A multimedia access 55 media access and display device produces audio and/or 
and display device 12 may be coupled to a communications video output signals to the user based at least in part on the 
network 14 by a communications path 16. Multimedia received stream. In the case of video output data, the video 
access and display device 12 may be any device for may be displayed on at least a first portion of a display. In 
receiving, processing and displaying audio and video the case of audio output data, the audio may be produced by 
streams, such as, for example, a personal computer (PC) or 60 at least one loudspeaker (not shown), 
other computer system having a display, a set-top box, an Subsequent to initiation of transmission of a stream of 
Internet appliance, a JAVA terminal, an information kiosk, or video data signals from a first source, the user may desire to 
a digital television (DTV), although the invention is not substantially simultaneously receive data signals from one 
limited in this respect. Communications network 14 may be or more additional sources. Additional sources may be 
any network for coupling computing devices such as, for 65 initiated and transmitted in a manner similar to the first 
example, a local area network (LAN), a wide area network source. However, additional processing may be employed to 
(WAN), an intranet, or the Internet, although the invention ensure that multiple sources are authored by multimedia 
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content providers, transmitted by signal sources, and Annex O of the H.263+ specification- Annex O defines three 

received by multimedia access and display devices in such scalability features termed temporal, signal to noise ratio 

a way as to provide substantially simultaneous display of the (SNR), and spatial scalability. With temporal scalability, an 

data signals from multiple sources on different portions of a encoder generates bi-directionally predicted frames (also 

display at a quality that is acceptable to the user. $ called true B frames). These frames support scalability in the 

Such processing may be best understood with reference to sense that they may be discarded without hindering the 

a specific example system having two active signal sources. decoding of surrounding frames in the stream. For example, 

However, the invention is not limited in scope to having only these frames maybe dropped at the decoder side to reduce 

two active signal sources and in some embodiments any the display frame rate (thus scaling processor usage), or they 

number of active signal sources may be handled according J0 may be dropped at the encoder side of a multi-point 

to the teachings of the present invention. FIG. 2 is a diagram connection, to free up bandwidth on low capacity commu- 

of a system 30 for dual source real-time video streaming nications channels. True B frames also offer the advantage 

over a network according to another embodiment of the that they may be compressed more efficiently than predicted 

present invention. In this embodiment of the present (p) frames, at approximately the same quality level, 

invention, there are two active signal sources, Source A 32 SNR scalability is a technique where the error introduced 

and Source B 34. Generally, these sources may be any b compression ^ itse lf encoded and transmitted as a 

systems having access to multimedia content. In one - ^ H263+ cncoded icturc A{ ^ receivin end mis 

embodiment, signal sources A 32 and B 34 may comprise ^ ennancement k icture be decoded and added back 

server computer systems coupled to computer network 36. ; than 

In another embodiment, the signal sources comprise a single , . r j j j o- ■ 

T * . j. ... r . . -JTi in a base picture was decoded. Since compression of the error 
server system. In one embodiment the computer network -20 t r . . . r 
may be the Internet. In various embodiments, the signali P lcturc 15 ltse * lossv > the remaimng error after that corn- 
sources may be structured to provide streaming layered P resslon ma y be interpreted as yet another picture, and may 
video streams authored to conform to International Tele- be compressed and transmitted as yet another enhancement 
communications Union (ITU) Video Coding for Low Bit layer- The H.263+ specification supports up to 14 enhance- 
Rate Communication, Recommendation H.263, version 2, 25 ment Uvers ^ manner > with the bandwidth allocated to 
1998 (also known as "H.263+"), and to accommodate net- each layer by an authoring application, 
work connections up to 56 Kbps. The H.263+ specification Spatial scalability is a variation of SNR scalability, where 
provides for scaleable video coding by allowing the total an enhancement layer may be four times as large (twice the 
bandwidth of a single stream to be divided into separate dimensions) as the lower layer. This allows a common 
layers of data for a given video signal source, each layer 30 interchange format (CIF) Input sequence, for example, to be 
corresponding to a different quality level associated with the encoded as both a base layer quarter common interchange 
video. Further information relating to layered streams may format (QCIF) sequence and an enhancement layer CIF 
be found in Annex O of the H.263+ specification. The sequence. Frame rate scalability is another feature available 
layered video encoding strategy employed by H.263+ imple- with SNR and spatial scalability. With frame rate scalability, 
mentations provides the capability to subscribe to primary 35 an enhancement layer may comprise more frames per sec- 
and secondary layers of a stream at different data rates, 0 nd than a base layer. 

thereby enabling a PIP mode of operation. [ n an embodiment of the present invention, the video 
Of course, the use of the H.263 specification to define the stream may comprise a plurality of layers including a base 
format of the stream and the 56 Kbps limit on the network layer (L0) at a data rate of 8 K BPS, supporting 5 frames per 
connection is only representative of various embodiments of 40 second (fps) of a 80 pixel by 60 pixel video image and an 
the present invention and the invention is not limited in intracoded frame-predicted frame-predicted frame (IP?) 
scope in these respects. profile; an enhancement layer 1 (LI, reference layer L0) at 
A client system 38 (such as a personal computer system, a data rate of 8 K BPS, providing a SNR layer, supporting 
for example) may be coupled to network 36 over a com- 5 fps of an 80 pixel by 60 pixel video image and an IPP 
munications path 40 as a multimedia access and display 45 profile; an enhancement layer 2 (L2, reference layer LI) at 
device. Communications path 40 comprises a fixed band- a data rate of 10 K BPS, providing a spatial layer, supporting 
width communications link (for example, a 56 Kbps link). 5 fps of a 160 pixel by 120 pixel video image and an IPP 
Client system 38 may comprise a decoder 42 for decoding profile; and an enhancement layer 3 (L3, reference layer L2) 
H.263+ layered streams received by the client and a graphi- at a data rate of 10 K BPS, providing a temporal layer, 
cal user interface 44 for displaying the video data signals of 50 supporting 15 fps of a 160 pixel by 120 pixel video image 
the streams to the user and accepting user input selections. and a multiple temporal scalability frame (BBBB) profile. 
Graphical user interface 44 may be used to select a desired- The scalability configuration described herein is an example 
multimedia content as a primary source for a first layer of a - of one embodiment and not the only stream configuration 
stream and another multimedia content as a secondary that may be used with the present invention. Furthermore, 
source for a second layer of the stream. Audio data associ- 55 the data rates, frame rates, and image sizes discussed above 
ated with both the primary and secondary sources may be are but one example of a layered stream configuration. One 
compressed for a resulting constant data rate of 8 Kbps in skilled in the art will appreciate that these parameters may 
this embodiment. The audio data signals may be transmitted be changed in other embodiments and this specific example 
to the client system on an audio stream. The configurations will not serve to limit the scope of the present invention, 
of layers used to encode video streams may be stored in eo According to this configuration of layers, the total band- 
catalog 46 that may be transmitted to client system 38 at the width used by the client system subscribing to the base layer 
time a multimedia transfer is initiated. The catalog may from one source and all base and enhancement layers from 
comprise subscription options and information regarding another source fits into the 56 K BPS communications path 
whether a particular stream may be used in a PIP mode of provided by communications path 40. This assumes that 
operation. 65 only one audio stream is subscribed to at any given point in 
In one embodiment, the individual video streams may be time. Generally, multiple audio streams are mutually exclu- 
authored to support configurations in accordance with sive and the audio data signals communicated to the client 
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system correspond to the primary source. In one exist, then multiple source streaming processing ends at 

embodiment, the audio stream may be included as a separate block 104. If at least one base layer exists for each source, 

layer within the stream. the client system subscribes to the base layers of the sources 

In this embodiment, the low bandwidth base layer (L0) at block 106, with each source providing data for a base 

may be used as a "preview" layer to provide PIP capability. 5 layer of the video stream. Next, at block 108, a primary 

FIG. 3 is a diagram illustrating the bandwidth partitioning of source may be selected for subscription and subsequent 

a communications path and corresponding video images processing of the enhancement layers associated with the 

according to one embodiment of the present invention. At primary source. At block 110, the audio data stream for the 

time Tl 50, a user of client system 38 selects the multimedia selected primary source may be subscribed to for subsequent 

content from signal source A 32 as the primary source of the 10 reception and production of audible sounds. After these 

stream and the multimedia content from signal source B 34 actions, the video stream being received by the client system 

as a secondary source of the stream. As a result of this comprises base layers for at least two signal sources and 

selection, video data from the primary source may be shown enhancement layers for one signal source. The client system 

on a display in a first portion 52 of the screen, and video data may also receive an audio stream corresponding to the signal 

from the secondary source may be shown on the display in 15 source of the enhancement layers. At block 112, the output 

a second portion 54 of the screen, thereby providing PIP signals for the audio stream and enhancement layers and 

capability. The first portion may be larger than the second base layer of the primary source may be produced for 

portion because more information for display is being reception by the user of the client system. This action may 

received for the primary source as compared to the second- comprise rendering the video data signals from the base and 

ary source. In this example, the secondary source data ("B") 20 enhancement layers of the video stream on a display and 

is shown in a rectangle in the lower right comer of the generating audible sounds through at least one loudspeaker 

screen. However, this is only an example and other sizes, from the audio data signals of the audio stream. The audible 

shapes, and placements may be used. sounds may be generated in synchronization with the dis- 

In the configuration of layers at time Tl, the client system played images from the base and enhancement layers. The 

subscribes to the enhancement layer 56 and a base layer 58 25 video data signals may be rendered to a first portion of the 

of signal source A 32, and a base layer 60 of signal source display. At block 114, the output signals for the base layer of 

B 34. Both base layers and the enhancement layers are secondary source may be produced for reception by the user 

transmitted along fixed bandwidth communications path 40. of the client system. This action may comprise rendering the 

The two decoded video sequences from these layers may be video data signals from the base layer to a second portion of 

displayed in the PIP format as shown. The client system also 30 the display, 

subscribes to an audio stream corresponding to the primary When the client system is in this configuration, the user 

source (at time Tl, this is from signal source A). At some may view a continual stream of data from a first multimedia 

time T2 62, the user may decide to switch the focus between content in a first portion of the display and a second 

the two windows. This may be indicated by double clicking multimedia content in a second portion of the display (e.g., 

a computer mouse button when a mouse pointer is over the 35 a PIP display). At some point in time, the user may desire to 

PIP area, for example, although any method of indicating change the selection of the primary source. For example, this 

input to the graphical user interface may be employed. As a could be accomplished by double clicking a mouse button 

result of the switch indication, the client changes the current with a mouse pointer over an area of the display or an 

subscription of sources to accept only the base layer 58 of equivalent keyboard action. When such an action is taken, 

signal source A, but to accept the base layer 60 and the 4Q the subscription for the selected portion of the display may 

enhancement layers 64 of signal source B. This results in a be changed to become the new primary source. At block 116, 

change to the display whereby video data signals from signal it may be determined if changing of the sources is desired 

source B are the primary source shown in the first portion 52 (e.g., by recognizing an appropriate input event). If no 

of the screen and video data signals from signal source A are change is indicated, then processing continues by producing 

the secondary source shown in the second portion 54 of the 45 more output signals from the received audio and video 

screen. The switch may be delayed by a small time interval streams at block 112. If a change is indicated, then the 

so that subscription changes may be synchronized with key enhancement layers of the primary source are unsubscribed 

frames in the stream that are assumed to be present at regular at block 118. At block 120, the client system subscribes to 

intervals. At the time of the switch, the audio subscription the enhancement layers of the secondary source. At block 

may also be changed to maintain the association with the 50 122, the identities of the primary and secondary sources are 

appropriate source selected as the primary source. switched or otherwise exchanged (i.e., the primary source 

At time T3 66, a user may desire to switch back to signal becomes the secondary source and vice versa). Processing 

source A as the primary source. As a result of the switch continues with block 110. 

indication, the client changes the current subscription of In an alternate embodiment of the present invention, the 

sources to accept only the base layer 60 of signal source B, 55 digital multimedia content may be authored as multiple 

but to accept the base layer 58 and the enhancement layers independent streams as opposed to a single stream having 

56 of signal source A. This results in a change to the display multiple layers. In one embodiment, the multiple indepen- 

whereby video data signals from signal source A are again dent streams may be authored according to the Smart- 

the primary source shown in the first portion 52 of the screen Stream™ streaming multimedia content technology com- 

and video data signals from signal source B are again the 60 mercially available from Real Networks, Inc., although the 

secondary source shown in the second portion 54 of the present invention is not limited in scope in this respect, 

screen. Unlike the layered approach used by the H.263+ specifica- 

FIG. 4 is a flow diagram of controlling bandwidth parti- tion to address bandwidth (and processor) scalability, Smart- 

tioning of multiple layers of a stream according to an Stream™ technology is a non-additive method of encoding 

embodiment of the present invention. After start 100, a client 65 a video sequence into a set of independent streams by 

system determines if base layers exist for at least two signal choosing a different set of encoding parameters for each 

sources at block 102. If a single base layer or no base layers stream. The streams are independent in the sense that each 
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stream may be decoded without any information from the systems comprising at least one processor, a data storage 
other streams. From the multimedia content author's point system (including volatile and non-volatile memory and/or 
of view, the author may choose to take the same video storage elements), at least one input device, and at least one 
content and encode it with different encoding parameters output device. Program code may be applied to input data to 
(e.g., data rate, frame rate, video size), storing all the 5 perform the functions described herein and generate output 
resultant streams in the same file. For example, one stream information. The output information may be applied to one 
may comprise base layer video data signals while another or more output devices, in known fashion. For purposes of 
stream may comprise enhancement layer video data signals. this application, a processing system includes any system 
The client system may switch between the different avail- that has a processor, such as, for example, a digital signal 
able streams under conditions of changing bandwidth (for 10 processor (DSP), a microcontroller, an application specific 
example, the client system may switch subscription from a integrated circuit (ASIC), or a microprocessor, 
high data rate stream to a lower data rate stream if the The programs may be implemented in a high level pro- 
transmission channel bandwidth decreases suddenly) and cedural or object oriented programming language to corn- 
processor load (for example, the client system may switch municate with a processing system. The programs may also 
from a high frame rate stream to a lower frame rate stream 15 be implemented in assembly or machine language, if 
to reduce the amount of processor usage in decoding the desired. In fact, the invention is not limited in scope to any 
video data signals). particular programming language. In any case, the language 

Referring back to FIG. 2, two active sources 32, 34 may may be a compiled or interpreted language, 

be used, with each source streaming an independent H.263+ xh c programs may be stored on a storage media or device 

stream according to Smarts tream™ video stream technol- 20 ( e . g>j hard disk drive, floppy disk drive, read only memory 

ogy. These streams may be authored to accommodate con- (ROM), CD-ROM device, flash memory device, digital 

nections up to a fixed data rate, such as 56 K BPS, for versatile disk (DVD), or other storage device) readable by a 

example. The client system 38 may be a personal computer general or special purpose programmable processing 

using software that can decode H.263+ SmartStream™ system, for configuring and operating the processing system 

video streams (e.g., decoder 42) and has a user interface 25 wnen the storage media or device is read by the processing 

(e.g., graphical user interface 44) to allow the user to select system to perform the procedures described herein. Embodi- 

one of the streams as the primary (main) stream and the ments of the invention may also be considered to be imple- 

other as a secondary (preview) stream. Audio associated mented as a machine-readable storage medium, configured 

with both streams in one example may be compressed at 8 f or ^ with a processing system, where the storage medium 

K BPS (constant bit rate streams). 30 so configured causes the processing system to operate in a 

The individual video streams representing digital multi- specific and predefined manner to perform the functions 

media content may be authored to support a configuration described herein. 

such as one having a first stream operating at 8 K BPS, and An example of one such type of processing system is 

supporting a 5 fps data rate of a 80 pixel by 60 pixel video ^ shown in FIG. 5. Sample system 400 may be used, for 

(e.g., a preview or base stream), and a second stream example, to execute the processing for methods in accor- 

operating at 28 K BPS supporting a 15 fps data rate of a 160 dance with the present invention, such as the embodiment 

pixel by 120 pixel video (e.g., a main or enhancement described herein. Sample system 400 is representative of 

stream). The SmartStream™ configuration described above processing systems based on the PENTIUM®, PENTIUM® 

is an example and not the only one that may be used for p r0j ^d PENTIUM® II microprocessors available from 

purposes of the present invention. In this embodiment, a low 4 T nte i Corporation, although other systems (including per- 

bandwidth stream may be used as the preview stream and the sona i computers (PCs) having other microprocessors, engi- 

total bandwidth used by a client system subscribing to the neering workstations, set-top boxes and the like) may also be 

preview stream from one source and the main stream from use d. In one embodiment, sample system 400 may be 

the second source still fits in a fixed bandwidth communi- ^ executing a version of the WINDOWS™ operating system 

cations channel (assuming only one audio stream is sub- available from Microsoft Corporation, although other oper- 

scribed to any given point in time). at ing systems and graphical user interfaces, for example, 

The configuration used to encode each video stream may may also be used, 

be stored in a catalog 46 that is transmitted to an application nG. 5 is a block diagram of a system 400 of one 

on the client system 38 at the time the multimedia transfer 5Q embodiment of the present invention. The computer system 

is initiated. This allows the client system to detect at the start 400 includes a processor 402 that processes data signals. The 

of the transfer what the subscription options are and to processor 402 may be a complex instruction set computer 

determine if a particular stream can be used in the manner (CISC) microprocessor, a reduced instruction set computing 

described by an embodiment of the present invention. (RISC) microprocessor, a very long instruction word 

In the preceding description, various aspects of the 55 (VLIW) microprocessor, a processor implementing a corn- 
present invention have been described. For purposes of bination of instruction sets, or other processor device, such 
explanation, specific numbers, systems and configurations as a digital signal processor, for example. FIG. 5 shows an 
were set forth in order to provide a thorough understanding example of an embodiment of the present invention imple- 
of the present invention. However, it is apparent to one mented as a single processor system 400. However, it is 
skilled in the art that the present invention may be practiced 60 understood that embodiments of the present invention may 
without the specific details. In other instances, well-known alternatively be implemented as systems having multiple 
features were omitted or simplified in order not to obscure processors. Processor 402 may be coupled to a processor bus 
the present invention. 404 that transmits data signals between processor 402 and 

Embodiments of the present invention may be imple- otner components in the system 400. 

mented in hardware or software, or a combination of both. 65 System 400 includes a memory 406. Memory 406 may be 

However, embodiments of the invention may be imple- a dynamic random access memory (DRAM) device, a static 

mented as computer programs executing on programmable random access memory (SRAM) device, or other memory 
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device. Memory 406 may store instructions and/or data may reside in another device such as a bus controller or other 

represented by data signals that may be executed by pro- controller device. Keyboard interface 424 allows coupling 

cesser 402. The instructions and/or data may comprise code of a keyboard to system 400 and transmits data signals from 

for performing any and/or all of the techniques of the present a keyboard to system 400. A user input interface 425 may be 

invention- Memory 406 may also contain additional soft- 5 coupled to the second I/O bus 420. The user input interface 

ware and/or data (not shown). A cache memory 408 may may be coupled to a user input device, such as a mouse, 

reside inside processor 402 that stores data signals stored in joystick, or trackball, for example, to provide input data to 

memory 406. Cache memory 408 in this embodiment speeds the computer system. Audio controller 426 may be coupled 

up memory accesses by the processor by taking advantage of to the second I/O bus 420. Audio controller 426 operates to 

its locality of access. Alternatively, in another embodiment, coordinate the recording and playback of audio signals. A 

the cache memory may reside external to the processor. bus bridge 428 couples first I/O bridge 412 to second I/O 

A bridge/memory controller 410 may be coupled to the brid S e 420 - ^ bus brid £ e operates to buffer and bridge data 

processor bus 404 and memory 406. The bridge/memory signals between the first I/O bus 412 and the second I/O bus 
controller 410 directs data signals between processor 402, 

memory 406, and other components in the system 400 and „ Embodiments of the present invention are related to the 

bridges the data signals between processor bus 404, memory ^ of ^ysttm 400 to receive multiple digital multimedia 

406, and a first input/output (I/O) bus 412. In some s ^™\ |; Accoriing to one embooirnent recepUon of user- 

.. 4 , r ., t_5 , ,, j selectable multiple digital multimedia streams may be per- 

embodiments the bridge/memory controller provides a ^ \ 1 * m m ^ {o ^ ^ 

graphics port for coupling to a graphics controller 413. In executing sequences of instructions in memory 406. Such 

this embodiment, graphics controller 413 interfaces to a M mstnictions may be read ^to memory 406 from another 

display device (not shown) for displaying images rendered comput er-readable medium, such as data storage device 422, 

or otherwise processed by the graphics controller 413 to a Qr from yia mc network 414) for 

user. The display device may comprise a television set, a example . ExeC ution of the sequences of instructions causes 

computer monitor, a flat panel display, or other suitable processor 402 to receive multiple digital multimedia streams 

display device. ^ acC ording to embodiments of the present invention. In an 

First I/O bus 412 may comprise a single bus or a com- alternative embodiment, hardware circuitry may be used in 

bination of multiple buses. First I/O bus 412 provides place of or in combination with software instructions to 

communication links between components in system 400. A implement embodiments of the present invention. Thus, the 

network controller 414 may be coupled to the first I/O bus present invention is not limited to any specific combination 

412. The network controller links system 400 to a network 30 of nardware circuitry and software, 

that may include a plurality of processing systems (not Tfae elements of system 400 perform their conventional 

shown in FIG. 3) and supports communication among mnctions well-known in the art. In particular, data storage 

various systems. The network of processing systems may devicc 422 may be ^ tQ provide long . term storage for thc 

comprise a local area network (LAN), a wide area network executable instructions and data structures for embodiments 

(WAN), the Internet, or other network. In some 35 of memo ds of reviving streaming multimedia data signals 

embodiments, a display device controller 416 may be ffom multiple S0U rces in accordance with the present 

coupled to the first I/O bus 412. The display device con- mvenuon , whereas memory 406 is used to store on a shorter 

troller 416 allows coupling of a display device to system 400 term bagis tfae execut able instructions of embodiments of the 

and acts as an interface between a display device (not methods for receiving streaming multimedia data signals 

shown) and the system. The display device may comprise a 40 ^ multipk S0Ufces m accordance with me present mven . 

television set, a computer monitor, a flat panel display, or ^ during execution by proceS sor 402. 

other suitable display device. The display device receives ^ ^ ^ beeQ describcd ^ rcfcrence tQ 

data signals from processor 402 through display device fltafi|ralive embodiments, this description is not intended to 

controller 416 and displays information contained in thedata be k a limiting sense . Various modifications of the 

signals to a user of system 400 45 embodiments, as well as other embodiments of 

In some embodiments, camera 418 may be coupled to the me invention? which are apparent to persons skilled in the art 

first I/O bus. Camera 418 may comprise a digital video {Q which ^ pertains are deemed t0 Ue the 

camera having internal digital video capture hardware that ^ ^ scope of me invention 

translates a captured image into digital graphical data. The What is claimed is* 

camera may comprise an analog video camera having digital 50 r A method of rece iving digital multimedia data signals 

video capture hardware external to the video camera for from multipk sources m a strcam communicated over a 

digitizing a captured image. Alternatively, camera 418 may fixed bandwidtn communications path comprising: 

comprise a digital still camera or an analog stffl camera subscribing t0 a ^ layer of a first ^vm, a first layer of 

coupled to image capture hardware. A second I/O bus 420 j J , , c ., c ' 

Y . - 1 , » • r tl . , a second source, and a second layer of the first source; 

may comprise a single bus or a combination of multiple 55 „ , . At _ L , 

buses. The second I/O bus 420 provides communication ™ c ™ m Z ?f signals corresponding to Uie subscribed 

links between components in system 400. A data storage ^ sccond m *° stream ovcr 

device 422 may be coupled to the second I/O bus 420. Hie fixed "^dwidth communications path; 

data storage device 422 may comprise a hard disk drive, a producing output signals corresponding to the received 

floppy disk drive, a CD-ROM device, a flash memory 6 o ^ s * nals for ^ sourcc 906 for thc 

device, or other mass storage device. Data storage device source; 

422 may comprise one or a plurality of the described data displaying the output signals for the first source in a first 

storage devices. portion of a display; 

A keyboard interface 424 may be coupled to the second substantially simultaneously displaying the output signals 

I/O bus 420. Keyboard interface 424 may comprise a 65 for the second source in a second portion of the display; 

keyboard controller or other keyboard interface device. unsubscribing the second layer of the first source; and 

Keyboard interface 424 may comprise a dedicated device or subscribing to a second layer of the second source. 
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2. The method of claim 1, wherein the first layer of the 
first source comprises a base layer of the stream, the second 
layer of the first source comprises an enhancement layer of 
the stream, and the first layer of the second source comprises 
another base layer of the stream. 

3. The method of claim 1, further comprising: 
subscribing to a third layer of the first source, the third 

Layer comprising audio data signals; 
receiving data signals corresponding to the third layer of 

the first source in a second stream over the fixed 

bandwidth communications path; and 
producing audio output signals corresponding to the 

received data signals for the third layer. 

4. The method of claim 1, wherein the second layer of the 
second source comprises an enhancement layer of the sec- 
ond stream. 

5. The method of claim 1, further comprising: 
displaying the output signals for the first source in the 

second portion of a display; and 
substantially simultaneously displaying the output signals 
for the second source in the first portion of the display. 

6. The method of claim 5, further comprising: 
unsubscribing the second layer of the second source; and 
subscribing to the second layer of the first source. 

7. The method of claim 6, further comprising: 
displaying the output signals for the first source in the first 

portion of a display; and 
substantially simultaneously displaying the output signals 
for the second source in the second portion of the 
display. 

8. An article comprising a machine readable medium 
having a plurality of machine readable instructions, wherein 
when the instructions are executed by a processor the 
instructions subscribe the processor to receive a first layer of 
a first source, a first layer of a second source, and a second 
layer of the first source, control the reception of data signals 
corresponding to the subscribed layers of the first and 
second sources in a stream over a fixed bandwidth commu- 40 
nications path, produce output signals corresponding to the 
received data signals for the first source and the second 
source, display the output signals for the first source in a first 
portion of a display, substantially simultaneously display the 
output signals for the second source in a second portion of 45 
the display, unsubscribe the second layer of the first source, 
and to subscribe the processor to a second layer of the 
second source. 

9. The article of claim 8, further comprising instructions 
to subscribe the processor to a third layer of the first source, 
the third layer comprising audio data signals, to receive data 
signals corresponding to the third layer in a second stream 
over the fixed bandwidth communications path, and to 
produce audio output signals corresponding to the received 
data signals for the third layer. 

10. The article of claim 8, further comprising instructions 
to display the output signals for the first source in the second 
portion of a display, and to simultaneously display the output 
signals for the second source in the first portion of the 
display. 

11. The article of claim 10, further comprising instruc- 
tions to unsubscribe the second layer of the second source; 
and to subscribe the processor to the second layer of the first 
source. 

12. A method of streaming digital multimedia data signals 
of multiple sources from at least one server system in a 
layered stream over a fixed bandwidth communications path 
in a network to a client system comprising: 
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subscribing to base layers of the multiple sources; 
subscribing to an enhancement layer of a selected one of 
the sources; 

receiving the base layers and the enhancement layer in the 
layered stream over the fixed bandwidth communica- 
tions path at the client system; 

displaying the base layer and the enhancement layer of the 
selected source in a first portion of a display of the 
client system; 

substantially simultaneously displaying the base layer of 

at least one non-selected source ia at least one other 

portion of the display; and 
unsubscribing the enhancement layer of the selected 

source and subscribing to an enhancement layer of a 

newly selected source. 

13. The method of claim 12, further comprising subscrib- 
ing to an audio layer of the selected source, receiving the 
audio layer at the client system, and producing audio output 
signals in synchronization with the displaying of the base 
and enhancement layers of the selected source. 

14. The method of claim 12, further comprising: 
receiving the base layers of the sources and the enhance- 
ment layer of the newly selected source; 

displaying the base layer and the enhancement layer of the 
newly selected source in the first portion of the display; 
and 

substantially simultaneously displaying the base layer of 
at least one non-selected source in at least one other 
portion of the display. 

15. The method of claim 12, further comprising control- 
ling subscription of the enhancement layer by a user of the 
client system by a graphical user interface. 

16. An article comprising a machine readable medium 
having a plurality of machine readable instructions, wherein 
when the instructions are executed by a processor of a client 
system the instructions subscribe the client system to base 
layers of multiple sources of multimedia data, subscribe the 
client system to an enhancement layer of a selected one of 
the sources, control the reception of the base layers and the 
enhancement layer in a layered stream over a fixed band- 
width communications path at the client system, display the 
base layer and the enhancement layer of the selected source 
in a first portion of a display of the client system, display the 
base layer of at least one non-selected source in at least one 
other portion of the display, unsubscribe the enhancement 
layer of the selected source anil subscribe to an enhancement 
layer of a newly selected source. 

17. The article of claim 16, further comprising instruc- 
tions for subscribing to an audio layer of the selected source, 
receiving the audio layer at the client system, and producing 
audio output signals in synchronization with the displaying 
of the base and enhancement layers of the selected source. 

18. The article of claim 16, further comprising instruc- 
tions for 

receiving the base layers of the sources and the enhance- 
ment layer of the newly selected source; 

displaying the base layer and the enhancement layer of the 
newly selected source in the first portion of the display; 
and 

substantially simultaneously displaying the base layer of 
at least one non-selected source in at least one other 
portion of the display. 

19. The article of claim 16, further comprising instruc- 
tions for controlling subscription of the enhancement layer 
by a user of the client system by a graphical user interface. 
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